ROCm and HIP: A Detailed 10-Chapter Tutorial: The Scientific Method for HIP Optimization

Optimization in the HIP environment must be treated as a rigorous empirical discipline rather than a series of intuitive guesses. By adopting a systematic workflow, developers ensure that every code modification is justified by data, moving performance engineering away from "optimization superstition" toward a repeatable, scientific cycle of hypothesis and verification.

The 6-Step Workflow

HIP performance guidelines recommend a systematic sequence:

Measure a baseline: Determine current execution time and throughput.
Profile the program: Use rocprofv3 to collect hardware counters.
Identify the bottleneck: Determine if you are compute-bound, memory-bound, or latency-bound.
Apply targeted optimizations: Focus only on the identified bottleneck.
Re-measure: Verify if the change actually improved performance.
Iterate: Repeat the process until goals are met.

Avoid Optimization Superstitions

Performance gains should be reproducible results from specific hardware interactions. Avoid these anti-patterns:

Changing kernel code before measuring current performance.
Tuning block size without knowing if the kernel is memory-bound.
Chasing occupancy numbers with no proof they matter to the specific workload.

TERMINAL bash — 80x24

> Ready. Click "Run" to execute.

QUESTION 1

What is the very first step in the HIP optimization scientific method?

Identify the primary hardware bottleneck.

Measure a baseline performance metric.

Apply loop unrolling to kernels.

Tune thread block sizes for maximum occupancy.

QUESTION 2

Which of these is considered an 'Optimization Superstition'?

Using profiling tools to check memory bandwidth.

Applying optimizations before verifying the bottleneck.

Iterating the process after re-measuring.

Matching data precision to hardware capabilities.

QUESTION 3

Why is chasing high occupancy numbers without proof often counterproductive?

Higher occupancy always leads to higher latency.

Occupancy doesn't matter for AMD architectures.

It may force the compiler to spill registers, reducing performance despite more active threads.

It prevents kernels from using HBM2 memory.

QUESTION 4

If you replace `float` with `double` and performance drops significantly, what have you likely identified?

A compute-bound bottleneck on FP32 units.

A host-side synchronization error.

A failure in the ROCm compiler JIT.

That block size tuning is mandatory.

QUESTION 5

What is the recommended tool for Step 2 (Profile the program) in modern ROCm environments?

gdb

rocprofv3

htop

amd-config

Case Study: Precision & Bottleneck Analysis

The Scientific Approach to Floating-Point Performance

A developer has a matrix multiplication kernel that currently uses `float`. They are following the 6-step HIP optimization workflow. During Step 3 (Identify the bottleneck), they decide to run an experiment by swapping all data types to `double` and re-measuring.

Replace `float` with `double` and compare performance. What are the expected results and what do they reveal about the hardware bottleneck?

Solution:
Replacing float (32-bit) with double (64-bit) typically reduces throughput by approximately 50% on hardware architectures (like CDNA/RDNA) that have fewer FP64 execution units compared to FP32. Furthermore, it doubles the memory bandwidth pressure because each element now requires 8 bytes instead of 4. If performance scales exactly with the throughput drop of the ALUs, the kernel is likely compute-bound. If it scales more closely with the doubling of data volume, it is likely memory-bound.

Why is this experiment better than simply 'guessing' that the kernel needs more occupancy?

Solution:
This experiment provides empirical data on how the kernel utilizes specific hardware subsystems (ALUs vs. Memory Bus). Chasing occupancy is a 'superstition' because high occupancy does nothing if the kernel is already saturating the HBM2 bandwidth or the FP32 pipeline. The scientific method ensures you only spend time optimizing the resource that is actually at its limit.